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ABSTRACT 

The present status of the Elementary and Secondary 
School Desegregation Study, funded by the United States Commission on 
Civil Rights and currently underway, was evaluated. The study is 
designed to investigate the relationship between type and extent of 
desegre- gation activity and the resultant level of desegregation in 
the schools. This study is valuable for a number of reasons. First, 
the existing research literature has not helped our understanding of 
this particular question. The contractor, Unicon, has made excellent 
efforts to improve on the data with which it was originally supplied. 
It has hired a specialist in individual desegregation plans to search 
court records and Department of Justice records for details of plans. 
Other data collection efforts should be to consider data from other 
sources and to conduct a phone survey and follow-up of districts for 
which available records do not indicate the presence of a plan. By 
concentrating on how established patterns of student and population 
movement are altered by intervening plans of different types, the 
Unicon analytical plans will be able to obtain estimates of policy 
effects that are reasonably free of the contamination of extraneous 
factors. Many specific aspects of the analysis have yet to be worked 
out, but none of them are unusual. The Unicon researchers are 
experienced, knowledgeable, and skilled. In summary, this is perhaps 
the most important research activity that the Commission could 
undertake. (KH) 
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Evaluation of School Desegregation Study 

Eric A. Hanushek 
Professor of Economics and Political Science 
University of Rochester 



Over the past several weeks I have evaluated the current state of 
the Elementary and Secondary School Desegregation Study that has been 
funded by the U.S. Commission on Civil Rights. This report details my 
views on the qualifications and capabilities of the current contractor 
on the condition of the data collection and analytical efforts, and on' 
the ultimate feasibility of the study given where it is now. 

It is my summary opinion that the analysis that is contemplated i« 
inherently important and will greatly expand our knowledge about key 
policy issues. Moreover, Unicon is particularly well-suited to do this 
analysis. They have made substantial progress in recovering from a 
very bad situation: Now there are reasonable prospects for doing useful 
analysis, where the same probably could not have been said six months 
ago. 



Background 

In January 1986 I was asked to provide an independent assessment 
of the school desegregation analysis that is currently underway. I had 
previously been aware of the existence of the study, and I had read 
several accounts of the public statements by Dr. Gary Orfield at the 
time of his resignation from the advisory panel for the study. 
However, until last month I had neither considered the study's 
objectives nor followed how the research was to be conducted. 

I have taken my charge from the Commission to be the conduct of « 
evaluation of the study plan and the prospects for developing useful 
analysis. This would include an evaluation of the research 
capabilities of the current contractor, Unicon Research Corporation. 
But, it would be entirely forward looking. It would not include any 
evaluation of the operation of the previous contractor (SDC) , of the 
deliberations or procedures of the Advisory Panel , or of other aspects 
of the history of this project except in so far as these might be 
relevant to the future prospects for the analysis. 

I am trained as an economist and have been involved in research 
and policy analysis related to elementary and secondary schools for 
close to twenty years. This involvement began with my participation in 
the Harvard seminar that studied the original "Coleman Report", 
Equality of Education al Opportunity, and has been continued in'my own 
research since then, while the focus of much of my own work has not 
been directly on integration or desegregation, such issues clearly 
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cannot be ignored in any systematic research into schools. Therefore, 
I have both followed research in this area and produced some related 
work. 

On January 30, 1986, Professor Peter Mieszkowski and I spent a day 
with the researchers at Unicon. This had followed a preliminary 
reading of the original RFP, of the SDC proposal, and of various 
intervening documents discussing changes in the analytical design. 
During the day at Unicon, we reviewed the current state of the data 
collection effort and discussed the future analysis plans. 

Nature of the Research 

The project is directed at answering a simple, but extremely 
important policy question: What is the effect on desegregation of 
using different policies to bring about that desegregation? Over the 
past two decades, a wide variety of desegregation plans have been 
applied to different school systems. These range from very rigid and 
detailed plans imposed by the courts to somewhat more amorphous plans 
voluntarily entered into by the districts themselves. The methods of 
achieving desegregation have also differed widely, including mandatory 
reassignment of virtually all students in a district, development of 
magnet schools, open enrollment programs of various types, and 
alterations of attendance bourxJaries. The USCCR study is designed to 
investigate the relationship between type and extent of desegregation 
activity and the resultant level of desegregation in the schools. 

Somewhat surprisingly, we do not have a good understanding of the 
ultimate effects of these different desegregation policies. Given the 
importance of the policy issue and the emotions that are involved, it 
seems natural to believe that researchers would have providec 
reasonably definitive answers as to the efficacy of different 
approaches to school desegregation. They have not. While a large body 
of research related to general questions of desegregation is available, 
very little of it has been directed at providing systematic evidence 
about the effectiveness of different types of desegregation plans. 

There are a variety of explanations for the lack of current 
understanding. Most importantly, the research is inherently difficult 
to do. Across school systems, desegregation efforts have occurred at 
different times, with varying degrees of local commitment, and with 
differential enforcement from the courts and federal agencies. The 
plans themselves differ in details that are difficult to describe in 
any systematic manner. And, school desegregation efforts are but one 
factor entering into the determination of where and under what 
circumstances children attend specific schools. The natural movement of 
employment and residential locations, the quality of particular 
teachers and schools, the costs of both schools and other public 
services, and the varying patterns of housing costs each affects 
observed school attendance factors. To be useful for policy 
deliberations, the research must be able to separate these various 
influences. 
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Existing studies have only looked at part of the general issue and 
have been the subject of considerable criticism on methodological 
grounds. The state of desegregation depends, in simplest terms, on the 
potential for interracial contact arrl the evenness of distribution of 
racial groups across a school system. The available studies terxi to 
concentrate on one or the other of these: The potential for contact at 
least in a dynamic sense is analyzed by looking at mobility patterns, 
or "white flight" as it is often labelled in this literature; the 
racial distribution of the school age population is usually measured 
and analyzed by indexes of desegregation. Separating these two 
research issues is not appropriate in the context of considering the 
efficacy of various policies toward desegregation of the schools. 
Further, the studies that have been done of each tend to concentrate on 
a limited set of factors that might explain the observed outcomes aixi 
with few exceptions have not identified the specific policy choices 
that are made in the area. The limited set of factors entering into 
the analysis will, without extreme care, lead to serious statistical 
problems that then bring any results into question. 

Data Requirements 



Since it is not possible to conduct scientific experiments that 
provide evidence about the effectiveness of the different policies, an 
essential part of any study is the systematic collection of data across 
a wide variety of school and nonschool circumstances. Natural 
variations in the behavior of individuals and schools and in the form 
of desegregation plans must provide the basis for separating the 
effects of different factors. Unfortunately, for the purposes of 
making reliable policy judgments, sufficient data on both desegregation 
outcomes and desegregation plans and on other characteristics of school 
districts have not been available. Constructing a data set that 
contains detailed and reliable information of this sort is one 
extremely valuable aspect of the current study . 

The data collection and the analysis must be carefully 
coordinated, and the reliability of any analytical results is highly 
dependent upon the quality of the data. It is essential that the 
sentple of school systems be representative of the population. The 
various kinds of desegregation plans must be characterized in ways that 
are suitable for the subsequent statistical analysis. The data must be 
carefully checked for reporting and coding errors. And, the 
information collected must match in scope the range of important 
influences on the structure of schools and location patterns. Each of 
these necessary features of the data collection effort is made 
difficult by the fact that the analysis is retrospective; certain 
things, such as matters that rely upon detailed recall of data by 
individuals, generally cannot be included because of reliability' 
problems. Furthermore, dealing with the problems associated with 
retrospective construction of a data base often requires sophisticated 
and artful statistical analysis. 
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The jninimal data requirements are for accurate school-by-school 
information on students ay race and year and for detailed descriptions 
of existing desegregation plans. Other kinds of data, such as detailed 
facts about employment patterns and changes of areas, would be useful, 
but their availability is not as crucial. The application of 
appropriate statistical analyses can frequently make up for missing 
data of this sort. 

Current State of the Data 

The Unicon effort has followed two major lines. First, 
considerable effort has gone into the preparation and cleaning of data 
provided by SDC. Second, a new effort to record and characterize 
desegregation plans has been begun. 

In my opinion, analysis based solely on the data originally 
supplied to Unicon would be impossible. The major problems relate to 
the development of data cm desegregation plans within separate 
districts. Individual school systems were asked to provide details of 
previous desegregation plars either through completion of a lengthy 
questionnaire or through interviews by a SDC employee who visited the 
system. The data sought were very detailed information about actions 
taken over the past fifteen years. These data, such as the numbers of 
students by race and grade involved in the pairing of schools under a 
1973 plan, are not normally kept by a school system. If they once had 
been recorded, it would be extraordinarily lucky if the current 
official receiving the survey questionnaire could locate the records. 
Simply put, comprehensive and reliable plan data cannot be collected in 
this manner. Even a cursory analysis of the responses by schools to 
the SDC survey indicates that most of these data are simply unusable, 

After seme initial efforts to clean these data on plans, Unicon 
made an entirely appropriate decision to collect most plan information 
in a different manner from different sources. The Unicon strategy has 
been to hire an individual (Dr. Michael Ross) who has extensive 
background in individual desegregation plans to search court records 
and Department of Justice records for the ietails of plans. While not 
without its problems, this appears to be an entirely sound strategy and 
one that offers reasonable chances of success. 

The largest potential problem with this approach is that some 
plans might be overlooked or incompletely recorded. Truly voluntary 
plans for districts cannot be found in this way. Also, some of the 
details—say the pairing of specific schools—may be hard to find in 
the written records. To some extent, missing the details can be 
circumvented through detailed analysis of data; that is, some of the 
events such as school pairings can be readily seen in the individual 
school records. By combining data from other sources, such as the 
Department of Education survey of the availability of magnet schools in 
different districts, external checks can be brought to bear. 
Nevertheless, plans for future data development should almost certainly 
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include provision for phone survey and follow-up of districts, 
particularly those districts for which available records do not 
indicate the presence of a plan- The most important issues to be 
addressed through such a phone effort would be whether or not plans 
were in existence in any year and the general type of desegregation 
instrumer > used. 

While there were numerous errors in the enrollment data set, these 
were not fatal and could be corrected. Such data cleansing efforts are 
expensive and require considerable sophistication anA skill. However, 
this process appears to be well under control and will be completed by 
Unicon soon. This in itself will be an important accomplishment since 
they will provide more accurate enrollment data than has been available 
and since these data have been central to many of the largest cind most 
systematic studies of school desegregation or white flight. 

While not yet complete, current efforts? to prepare a usable data 
set are well on their way and promise to produce the best data set ever 
available for understating the course ejxI pattern of school 
desegregation. 

Analysis Plans 

Previous research provides some Insights into the patterns of 
racial isolation in the schools. Into the differences that evolve over 
time due to movements of the population and due to court judgments, and 
into the importance of various community factors such as attitudes, 
past history, and the like. However, in part owing to the 
unavailability of suitable data, these analyses have fallen short of 
providing reliable and systematic answers to basic policy questions 
about the effects of specific instruments used in bringing about school 
desegregation. 

The Unicon analytical plans, while not completely detailed at this 
time, center on understanding the dynamics of population movement and 
school changes. By concentrating (in a statistical sense) on how 
established patterns of student and population movement are altered by 
intervening plans of different types, they will be able to obtain 
estimates of policy effects that are reasonably clear of the 
contamination of other, extraneous factors. Further, in many cases 
they will be able to identify specific segments of a school system that 
are directly affected by desegregation plans and other segments that 
are not directly affected; this provides a natural "control group" that 
can be exploited to refine estimates of the independent effects of 
desegregation efforts. 

There are many specific aspects of the analysis yet to be worked 
out. For example, past researchers have used a variety of summary 
statistical measures that are designed to capture the character of the 
racial distribution across schools. Some decisions will need to be 
made about which of the measures to apply. (Presumably a variety will 
ultimately be analyzed both to link this work to previous analyses and 
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to understand whether or not the results are particularly sensitive to 
technical measurement issues). Other matters, such as the appropriate 
way for combining districts of different types (by size, location, 
etc.) must be developed along the way. However, these are the normal 
types of issues that are addressed in any analysis. 

Evaluation of Unicon Research Corporation 

The researchers at Unicon have extensive analytical experience, 
are very familiar with the existing research, aixJ have indicated a 
degree of sophistication well beyond that of most previous researchers 
in this area. Therefore, there is every reason to believe that this 
analysis, when completed, will become the benchmark for understanding 
school desegregation policies. 

Unicon Research Corporation is relatively small, but it has a very 
powerful analytical capability. It was founded by Dr. Finis Welch, a 
truly notable economist with extensive experience in statistical 
analysis. Dr. Welch and his co-workers have experience not only with 
relevant statistical work but with data collection and preparation. 
For example, Unison was asked by the National Science Foundation to 
analyze the sampling and reliability of the Panel Study on Income 
Dynamics, a major data resource for social scientists. 

The obvious deficiency of the Unicon researchers is background in 
the specifics of desegregation plans, but that deficiency has been 
corrected by hiring Dr. Michael Ross to research and prepare plan data. 
Dr. Ross is one of the most experienced researchers in the area of 
desegregation plans and combines an academic background with many years 
of direct involvement in desegregation matters. Combined with the 
knowledge and experiences of the members of the Advisory committee, 
this lack of background by Unicon appears to be adequately compensated 
for. 

The primary analytical tasks are ones that Unicon is very well 
suited for. They involve the statistical analysis of complicated data 
sets in itfuch the dynamic structure is particularly important . These 
are the types of casks that applied eeonometricians are best able to 
deal with (as compared to other social scientists or applied 
statisticians). Among applied eeonometricians, Dr. Welch is one of the 
best. This is evidenced by his extensive publication record, by his 
election as a Fellow of the Econometric Society, and by his service on 
various editorial boards, advisory committees, and the like. 

Previous involvement in the analysis of desegregation issues is 
most critical when it canes to developing the data base for existing 
desegregation plans ir. different districts. Such experience is much 
less important when it comes to the actual analysis. During the 
analytical phase, experience with statistical methodology, the handling 
of large panel data sets, and so forth is much more critical. Unicon 
seems well situated for this analytical phase. 
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Summary Recommendations 

In my opinion, this is perhaps the most important research 
activity that the U.S. Commission of Civil Rights could undertake. The 
process of school desegregation is one of the most important issues of 
public policy of the past three decades. Yet, our understanding of the 
impacts of different policies is by any standards remarkably primitive. 

The current study would enhance considerably our understanding. 
It is developing an important new data set that will allow a wide 
variety of investigations to be conducted. The analytical plans are 
sound. And, the policy questions are central to the entire area. 

This research is clearly expensive, but the value is also high. 
The development of reliable and comprehensive historical data bases is 
a difficult and time consuming task. Nevertheless, since such weighty 
decisions as the integration of our students and society relate 
directly to the policy analyses contained in this study, it would seem 
to be very short sighted to discontinue such a promising endeavor at 
this time. 

This review has been motivated by concerns related for the 
potential of biases in the conclusions — biases arising either from 
specific political views or from incomplete analyses. This does not 
appear to be a major problem. First, the researchers currently 
involved have established a lengthy record of impartial analysis of 
important public policies. Second, any results and analyses by them 
will be subject to intense scrutiny. Third, even further safeguards 
can be build in by insuring that the data set is made publicly 
available to other researchers. This sort of check has been used 
extensively in scientific investigations to insure that the conclusions 
of analysis are appropriate. 

There is no reason to believe that standard scientific checks on 
any research will be any less effective in this case than they are in 
general. In fact, because of the sensitive and emotional nature of the 
topic, this analysis is likely to receive much more ©ctensive and 
timely attention than most social science analyses. Past history 
provides numerous examples of this scientific mechanism at work. One 
relevant case is the "Coleman Report" where the controversial nature of 
the policy conclusions generated extensive re-analysis and parallel 
analyses that continues even today. This type of rjatural reaction 
serves to protect us from erroneous policy conclusions. 



